GUI Testing: Pitfalls and Process

G raphical user interfaces have become a nearly ubiquitous means of interacting with software systems. The GUI responds to user events, such as mouse movements or menu selections , providing a front end to the underlying application code. The GUI interacts with the underlying code through messages or method calls. GUIs make software easy to use, and developers are dedicating a larger portion of code to implementing them. GUIs can constitute as much as 60 percent of an application's total code today. The use of GUIs in safety-critical systems is also growing, making their correct operation imperative. Given their increased importance, testing GUIs for correctness can enhance the entire system's safety, robust-ness, and usability Current GUI testing techniques are incomplete, ad hoc, and largely manual. The most common tools use record-playback techniques. A test designer interacts with the GUI, generating mouse and keyboard events. The tool records the user events, captures the GUI session screens, and then stores the session—usually as a script. The tester later plays back the recorded sessions to recreate the events with different inputs. This process is extremely labor intensive , often relying on the test designer's ability to generate interesting GUI interactions. An automatic test case generator can provide a higher level of support, but a programmer must code it for all possible decision points in the GUI. Automated or not, the time-consuming record-playback approach easily misses important GUI decisions. A popular alternative is to release beta copies of the software and let the users do part of the testing. For example, Microsoft tested part of its Windows 95 software by releasing almost 400,000 beta copies. Software testing is already labor and resource intensive—often accounting for 50 to 60 percent of total software development costs—and GUI testing poses further difficulties that traditional software testing techniques do not adequately address. Conventional software uses coverage criteria as a guideline for determining testing adequacy. These criteria define sets of rules that test designers use to determine the type and amount of underlying code to test. For example , a criterion for " event coverage " might require all reachable events in a GUI to execute at least once during a complete cycle of test cases. However, traditional coverage criteria do not work well for GUIs. First, GUI software differs from the underlying application code in its level of abstraction, so mapping between GUI events and the underlying code …